Apparatus and method for image processing and storage medium for the same

ABSTRACT

An apparatus and a method as well as a storage device of the present invention transform intensity levels with consideration of both features of the intensity levels in an input image and human visual characteristics, wherein the present invention includes the steps of inputting wider dynamic range digital image data, generating a histogram of intensity levels of the image data, generating a histogram equalization LUT by cumulating the histogram, generating a visual characteristics LUT with reference to the histogram equalization LUT, generating a combined LUT by combining the histogram equalization LUT and the visual characteristics LUT, and correcting the image data.

CROSS REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 09/970,579 filed on Oct. 4, 2001, now U.S. Pat. No. 6,996,271the disclosure of which is herein incorporated by reference.

BACKGROUND OF THE INVENTION

The present invention relates to an apparatus and a method for imageprocessing as well as a storage medium for the same, and particularly toan apparatus and a method for image processing as well as a storagemedium that are preferably used for transforming a dynamic range ofintensity levels of pixels forming an image.

As the performance of CCDs (Charge Coupled Devices) installed inelectronic apparatuses that have the ability to acquire image data, suchas digital cameras, scanners and the like, has become higher, agradation dynamic range of images tends to become wider.

However, when images having a wider gradation dynamic range thanconventional one (hereinafter referred to as “wider dynamic rangeimages”) are supplied to reproducing apparatuses such as displays,printers and the like, which correspond to images having conventionaldynamic range (hereinafter referred to as “narrower dynamic rangeimages”), there is a need for a gradation correction technique thattransforms the wider dynamic range images into the narrower dynamicrange images.

As one of conventional gradation correction techniques, a histogramequalization method is well known. The histogram equalization methodtransforms pixels' intensity levels so that the intensity levels aredistributed uniformly over an entire image.

A basic algorithm for the histogram equalization method will be nowdescribed. As shown in FIG. 1A, a histogram showing a frequency of everyintensity level for an input image (an image before transformation), ahistogram showing the number of pixels having identical intensity levelsis generated. Assuming that a frequency of an intensity level Xn isH(Xn) (n=0, 1, 2, . . . , max), the histogram thereof is cumulated. Morespecifically, a cumulative frequency ΣH(X0) for an intensity level X0 isset to H(X0); a cumulative frequency ΣH(X1) for an intensity level X1 isobtained by calculation of [H(X0)+H(X1)]; a cumulative frequency ΣH(X2)for an intensity level X2 is obtained by calculation of[H(X0)+H(X1)+H(X2)]; and a cumulative frequency ΣH(Xmax) for anintensity level Xmax is obtained by calculation of [H(X0)+H(X1)+ . . .+H(Xmax)]. As a result of such cumulation, a cumulative histogram isgenerated. Further, a look up table (hereinafter referred to as “LUT”)shown in FIG. 1B is generated by adapting a dynamic range (X′0-X′max) ofintensity levels of an output image to a range from the cumulativefrequency ΣH(X0) for the intensity level X0 to the cumulative frequencyΣH(Xmax) for the intensity level Xmax (the vertical axis of thecumulative histogram).

When the intensity levels of the input image are transformed using theLUT generated in this manner, the resultant output image has a uniformfrequency (an even distribution) of its intensity levels as shown in ahistogram of FIG. 1C and therefore has a more enhanced contrast.

However, when such histogram equalization method is adapted to an inputimage that has an extreme peak of its intensity frequency, intensitylevels near the peak may be transformed into a wider range within thedynamic range of a resultant output image, and therefore a desiredresult may not be obtained (the resultant image may be indistinct).

In addition, in the conventional gradation correction technique, sincehuman visual characteristics, that is, more particularly,Weber-Fechner's law stating that “human sensation is proportional to thelogarithm of stimulus intensity” are not considered, the narrowerdynamic range image as a result of the transformation may be indistinct.

SUMMARY OF THE INVENTION

In an embodiment, the present invention provides intensitytransformation that can be performed considering both intensitydistribution characteristics of input images and the human visualcharacteristics.

According a first embodiment of the present invention, there is providedan image processing apparatus for correcting a gradation of an inputimage data, including: a first generating section for generating a firstlook up table using the input image data; a second generating sectionfor generating a second look up table based upon the first look up tableand human visual characteristics; a combining section for generating athird look up table by combining the first and second look up tablesaccording to a predetermined combining ratio; and a transforming sectionfor transforming the image data using the third look up table.

In an embodiment, the first generating section may generate the firstlook up table based upon a histogram of intensity levels of the imagedata.

In an embodiment, the second generating section may generate the secondlook up table using a predetermined logarithm curve as the human visualcharacteristics.

In an embodiment, the image processing apparatus, preferably, furtherincludes an input section for inputting the combining ratio.

In an embodiment, the image processing apparatus, preferably, furtherincludes a setting section for setting the combining ratio based uponthe first and second look up tables.

In an embodiment, the transforming section may transform a dynamic rangeof the intensity levels of the image data using the third look up table.

According to a second embodiment of the present invention, there isprovided an image processing method, wherein an image processingapparatus is used for correcting a gradation of an input image data, themethod including the steps of generating a first look up table using theinput image data, generating a second look up table based upon the firstlook up table and human visual characteristics, generating a third lookup table by combining the first and second look up tables according to apredetermined combining ratio, and transforming the image data using thethird look up table.

According to a third embodiment of the present invention, there isprovided a storage medium in which a computer readable program for imageprocessing to correct a gradation of an input image data, the programincluding the steps of generating a first look up table using the inputimage data, generating a second look up table based upon the first lookup table and human visual characteristics; generating a third look uptable by combining the first and second look up table according to apredetermined combining ratio, and transforming the image data using thethird look up table.

According to a fourth embodiment of the present invention, there isprovided an image processing apparatus for correcting a gradation of aninput image data, the apparatus including a first generating section forgenerating a first look up table using the input image data, a secondgenerating section for generating a second look up table based upon thefirst look up table and human visual characteristics, and a transformingsection for transforming the image data using the second look up table.

In an embodiment, the first generating section may generate the firstlook up table based upon a histogram of intensity levels of the imagedata.

In an embodiment, the second generating section may generate the secondlook up table using a predetermined logarithm curve as the human visualcharacteristics.

In an embodiment, the transforming section may transform a dynamic rangeof the intensity levels of the image data using the second look uptable.

According to a fifth embodiment of the present invention, there isprovided an image processing method of an image processing apparatus forcorrecting a gradation of an input image data, the method including thesteps of generating a first look up table using the input image data,generating a second look up table based upon the first look up table andhuman visual characteristics, and transforming the image data using thesecond look up table.

According to a sixth embodiment of the present invention, there isprovided a storage medium in which a computer readable program for imageprocessing to correct a gradation of an input image data, the programincluding the steps of generating a first look up table using the inputimage data, generating a second look up table based upon the first lookup table and human visual characteristics, and transforming the imagedata using the second look up table.

In the image processing apparatus and method as well as the programaccording to the first, second, and third embodiment of the presentinvention, a first look up table that has been generated using inputimage data and a second look up table that has been generated based uponthe first look up table and human visual characteristics are combinedaccording to a predetermined combining ratio to generate a third look uptable, and then the image data is transformed using the third look uptable, so that it is possible to transform intensity levels withconsideration of both the features of distribution of the intensitylevels in the input image and the human visual characteristics.

Further, in the image processing apparatus and method as well as theprogram according to the fourth, fifth, and sixth embodiments of thepresent invention, a first look up table is generated using input imagedata, a second look up table is generated based upon the first look uptable and human visual characteristics, and then the image data istransformed using the second look up table, so that it is possible totransform intensity levels with consideration of both the features ofdistribution of the intensity levels in the input image and the humanvisual characteristics.

Additional features and advantages of the present invention aredescribed in, and will be apparent from, the following DetailedDescription of the Invention and the figures.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1A through 1C show diagrams illustrating a histogram equalizationmethod.

FIG. 2 shows a block diagram of an imaging apparatus in accordance withthe principles of the present invention.

FIG. 3 shows a block diagram of an image processing section inaccordance with the principles of the present invention.

FIG. 4 shows a diagram illustrating Weber-Fechner's law.

FIG. 5 shows a flow chart describing an operation of an image processingsection in accordance with the principles of the present invention.

FIG. 6 shows a flow chart illustrating a process for generating a visualcharacteristics LUT in Step S4 of FIG. 5.

FIG. 7 shows a diagram illustrating a process for generating a combinedLUT.

FIG. 8 shows a block diagram of an image processing section inaccordance with the principles of the present invention.

FIG. 9 shows a block diagram of an image processing section inaccordance with the principles of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 2 shows an imaging apparatus in accordance with an embodiment ofthe present invention. Preferably, the imaging apparatus acquires anoptical image of an object as wider dynamic range (e.g., 16-bit wide)image data and transforms it into narrower dynamic range (e.g., 8-bitwide) image data appropriately.

In an embodiment, an optical system of the imaging apparatus includes: alens 1 that collects the optical image of the object; an aperture 2 thatadjusts a quantity of light of the optical image; a shutter 3 thatadjusts duration of incidence of the optical image; an infrared cutofffilter (IR filter) 4 that filters out light in infrared region to whicha CCD 7 is too sensitive; and a low pass filter (LPF) 5 that rejectshigher optical frequency components to suppress aliasing as a result ofdiscrete sampling by the CCD 7. It should be noted that the infraredcutoff filter 4 may be omitted by applying a coating for rejecting theinfrared region to the lens 1.

Further, the imaging apparatus includes: a timing generator 6 thatgenerates horizontal scanning timing and vertical scanning timing forthe CCD 7; the CCD 7 that outputs wider dynamic range image data to aCDS 8 by photoelectrically converting the optical image input via theoptical system; the CDS (correlated double sampling) 8 that reducesnoise by sampling the image data input from the CCD 7; an AGC (automaticgain controller) 9 that electrically amplifies the amplitude of theimage data depending on the brightness of the object; an A/D converter10 that digitizes the analog image data; an image processing section 11that comprises a DSP (digital signal processor) and so on and convertsthe wider dynamic range digital image data into narrower dynamic rangedigital image data; and a controlling section 12 that controls a drive14 to read a controlling program stored in a magnetic disk 15, anoptical disk 16, a magneto-optical disk, or a semiconductor memory 18and further controls the entire imaging apparatus based upon thecontrolling program read through the drive 14, user commands inputthrough a manipulating section 13 and the like.

In the imaging apparatus, the optical image that is input via theoptical system including a series of components from the lens 1 to thelow pass filter 5 is converted into the wider dynamic range image databy the CCD 7, undergoes the noise reduction process by the CDS 8 and theamplitude amplification by the AGC 9, and is further digitized by theA/D converter 10 and input into the image processing section 11. TheWider dynamic range digital image data that is input into the imageprocessing section 11 is transformed into the narrower dynamic rangedigital image data.

FIG. 3 shows an embodiment of image processing section 11 (see FIG. 2).Preferably, an image memory 31 stores the wider dynamic range digitalimage data input from the A/D converter 10. A histogram generatingsection 32 generates a histogram showing distribution of intensitylevels in the image data read from the image memory 31 and stores it ina histogram memory 33. A histogram equalization LUT generating section34 calculates a cumulative histogram by cumulating the histogram readfrom the histogram memory 33 in a manner similar to the process forgenerating a LUT of FIG. 1B from a histogram of FIG. 1A, and furtheradapts a narrower dynamic range to the vertical axis of the cumulativehistogram in order to generate a histogram equalization LUT and store itin the histogram memory 33.

A visual characteristics LUT generating section 35, referring to thehistogram equalization LUT stored in the histogram memory 33, generatesanother LUT with consideration of human visual characteristics(hereinafter referred to as a “visual characteristics LUT”) and storesit in a visual characteristics memory 36. The human visualcharacteristics and the process for generating the visualcharacteristics LUT will be described in detail later.

A LUT combining section 37 combines the histogram equalization LUT readfrom the histogram memory 33 and the visual characteristics LUT readfrom the visual characteristics LUT memory 36 according to a combiningratio read from a combining ratio memory 38 and stores the resultantcombined LUT in a combined LUT memory 39. Preferably, the combiningratio memory 38 stores the combining ratio that has been set by the userusing the manipulating section 13.

An image data correction section 40 transforms intensity levels of thewider dynamic range image data read from the image memory 31 using thecombined LUT read from the combined LUT memory 39 and outputs theresultant narrower dynamic range image data as corrected data.

FIG. 4 shows a diagram illustrating Weber-Fechner's law. Weber-Fechner'slaw states that “human sensation is proportional to the logarithm ofstimulus intensity.” That is, assuming that the stimulus intensity is Rand the sensation is E, the following relational expression isestablished:

Sensation E=K*log R where K is a predetermined constant.

FIG. 5 shows a flow chart describing an operation of an imagingprocessing section 11 (see FIG. 3) in accordance with an embodiment ofthe present invention. In Step S1, wider dynamic range digital imagedata is input from the A/D converter 10 (see FIG. 2) to the imageprocessing section 11. The input image data is stored in the imagememory 31. In Step S2, the histogram generating section 32 reads theimage data from the image memory 31 and generates a histogram of theimage data's intensity levels and stores it in the histogram memory 33.In Step S3, the histogram equalization LUT generating section 34 readsthe histogram from the histogram memory 33 to calculate a cumulativeLUT, and then generates a histogram equalization LUT by adapting anarrower dynamic range to the vertical axis of the cumulative histogramand stores it in the histogram memory 33.

In Step S4, the visual characteristics LUT generating section 35generates a visual characteristics LUT by referring to the histogramequalization LUT stored in the histogram memory 33 and stores it in thevisual characteristics memory 36. A process for generating the visualcharacteristics LUT will be described with reference to the flow chartof FIG. 6 as well as FIG. 7. It is noted that FIG. 7 shows a curve forthe histogram equalization LUT, a curve for the visual characteristicsLUT and a curve for the combined LUT.

In Step S11, the visual characteristics LUT generating section 35 setstwo thresholds N1, N2 for determination higher and lower noise levelsfor the narrower dynamic range. The thresholds may be set, for example,based upon the shape of the curve of the histogram equalization LUT (thecumulative histogram), or it may be set to any predetermined fixedvalue, or it may be also set by the user.

In Step S12, the visual characteristics LUT generating section 35determines for the histogram equalization LUT the minimum value “low”that is the lowest value in the wider dynamic range that corresponds tothe values higher than the threshold for the lower noise level N1 in thenarrower dynamic range, and determines a point which value is “low” inthe wider dynamic range and the minimum (“zero”) in the narrower dynamicrange as P1.

In Step S13, the visual characteristics LUT generating section 35determines for the histogram equalization LUT the maximum value “high”that is the highest value in the wider dynamic range that corresponds tothe values lower than the threshold for the higher noise level N2 in thenarrower dynamic range, and determines a point which value is “high” inthe wider dynamic range and the maximum (“max”) in the narrower dynamicrange as P2.

In Step S14, the visual characteristics LUT generating section 35calculates a logarithm curve y that passes through the points P1, P2 asthe visual characteristics LUT:y=a*log(x)+b

where x, y designate values of the input intensity (the horizontal axis)and the output intensity (the vertical axis), respectively.

Now, referring back to FIG. 5, in Step S5, the LUT combining section 37reads the histogram equalization LUT (hereinafter also referred to asLUT1(x)) from the histogram memory 33, reads the visual characteristicsLUT (hereinafter also referred to as LUT2(x)) from the visualcharacteristics LUT memory 36, reads the combining ratio from thecombining ratio memory 38, and then calculates a combined LUT(hereinafter also referred to as LUT3(x)) according to the followingexpressions and stores it in the combined LUT memory 39:LUT3(x)=0 (0<x<low)LUT3(x)=α*LUT1(x)+(1−α)*LUT2(x) (low≦x≦high)LUT3(x)=max (high<x≦max)

In Step S6, the image data correcting section 40 reads the wider dynamicrange digital image data from the image memory 31, transforms theintensity levels of the wider dynamic range digital image data using thecombined LUT read from the combined LUT memory 39 and outputs theresultant narrower dynamic range image data as the corrected data.

As described above, in the image processing section 11, the widerdynamic range digital data is transformed into the narrower dynamicrange digital image data by using the combined LUT into which thehistogram equalization LUT based upon the histogram of the intensitylevels of the wider dynamic range image data and the visualcharacteristics LUT with consideration of the-human visualcharacteristics are combined.

FIG. 8 shows another embodiment image processing section 11. Preferably,a combining ratio computing section 51 has been added to the imageprocessing section 11, wherein the combining ratio computing section 51computes the combining ratio by analyzing features of the histogramequalization LUT read from the histogram memory 33 and the visualcharacteristics LUT read from the visual characteristics LUT memory 36and stores it in the combining ratio memory 38.

The operation of this embodiment of the image processing section 11 issimilar to the image processing station shown in FIG. 3, except that aprocess for computing the combining ratio by the combining ratiocomputing section 51 is added to the process described with reference tothe flow chart of FIG. 5 between the process of Step 4 (the process forgenerating the visual characteristics LUT) and the process of Step 5(the process for combining both LUTs), and therefore its description isomitted.

FIG. 9 shows another embodiment of the image processing section 11.Preferably, the LUT combining section 37, the combining ratio memory 38,and the combined LUT memory 39 have been deleted from the imageprocessing section shown in FIG. 3.

Preferably, the image correcting section 40 transforms the intensitylevels of the wider dynamic range image data using the visualcharacteristics LUT read from the visual characteristics LUT memory 36to acquire the narrower dynamic range image data.

In an embodiment, it is also possible to apply the present invention tothe case in which the intensity levels are altered without alteration ofthe dynamic range of images.

In an embodiment, it is also possible to apply the present invention notonly to the imaging apparatus in the embodiments described above, butalso to other electronic apparatuses that process image data, such asscanners, facsimile machines, copying machines and the like.

A series of the above-described processes may be performed by hardware,though it also may be performed by software in further embodiments ofthe present invention. Preferably, when the series of the processes isperformed by software, a program constituting the software and stored ina storage medium shall be installed in a computer incorporated indedicated hardware, or in any other computer that can perform variousfunctions by installing various programs, such as, for example, ageneral-purpose personal computer and the like.

The storage medium may, as shown in FIG. 2, include package media thatmay include magnetic disks 15 (including floppy disks), optical disks 16(including CD-ROM (compact disk read only memory) and DVD (digitalversatile disks)), magneto-optical disks 17 (including MD (Mini Discs)),or semiconductor memory 18 and the like, in which the program is storedand which are supplied to provide the user with the program, orotherwise it may include ROM or hard disk drives, in which the programis stored and which is supplied to the user, being pre-installed in thecomputer.

In this specification, it should be noted that the steps describing theprogram stored in the storage medium may include not only the processesthat are performed chronologically in the order in which they have beendescribed, but also the processes that are performed not alwayschronologically, but in parallel or independently.

It should be understood that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications can be madewithout departing from the spirit and scope of the present invention andwithout diminishing its intended advantages. It is therefore intendedthat such changes and modifications be covered by the appended claims.

1. An image processing apparatus for correcting a gradation of inputimage data, comprising: a first generating section for generating afirst look up table using the input image data; a second generatingsection for generating a second look up table based on the first look uptable and human visual characteristics, wherein the human visualcharacteristics are generated by setting a low point and a high point onan output intensity versus input intensity graph and a logarithm curveis calculated between the high and low points; and a transformingsection for transforming the image data using the second look up table,wherein the first generating section generates the first look up tablebased on a cumulative histogram of intensity levels of the image data.2. An image processing apparatus as claimed in claim 1, wherein thesecond generating section generates the second look up table using apredetermined logarithm curve as the human visual characteristics.
 3. Animage processing apparatus as claimed in claim 1, wherein thetransforming section transform a dynamic range of the intensity levelsof the image data using the second look up table.
 4. An image processingmethod of an image processing apparatus for correcting a gradation ofinput image data, the method comprising the steps of: generating a firstlook up table using the input image data, wherein the input image datais for a wider dynamic range image; generating a second look up tablebased on the first look up table and human visual characteristics,wherein the human visual characteristics are generated by setting a lowpoint and a high point on an output intensity versus input intensitygraph and a logarithm curve is calculated between the high and lowpoints; and transforming the image data using the second look up tableto produce a narrower dynamic range image.
 5. An image processing methodas claimed in claim 4, wherein the first generating step generates thefirst look up table based on a histogram of intensity levels of theimage data.
 6. An image processing method as claimed in claim 4, whereinthe second generating step generates the second look up table using apredetermined logarithm curve as the human visual characteristics.
 7. Animage processing method as claimed in claim 4, wherein the transformingstep transforms a dynamic range of the intensity levels of the imagedata using the second look up table.
 8. A storage medium for storing acomputer readable program for image processing to correct a gradation ofinput image data, the program comprising: a first generating step ofgenerating a first look up table using the input image data; a secondgenerating step of generating a second look up table based on the firstlook up table and human visual characteristics, wherein the secondgenerating step generates the second look up table using a predeterminedlogarithm curve as the human visual characteristics and the logarithmcurve is determined by setting a low point and a high point on an outputintensity versus input intensity graph; and a transforming step oftransforming the image data using the second look up table.
 9. A storagemedium as claimed in claim 8, wherein the first generating stepgenerates the first look up table based on a histogram of intensitylevels of the image data.
 10. A storage medium as claimed in claim 8,wherein the transforming step transforms a dynamic range of theintensity levels of the image data using the second look up table.
 11. Astorage medium for storing a computer readable program for imageprocessing to correct a gradation of input image data, the programcomprising: a first generating step of generating a first look up tableusing the input image data; a second generating step of generating asecond look up table based on the first look up table and human visualcharacteristics, wherein the second generating step generates the secondlook up table using a predetermined logarithm curve as the human visualcharacteristics; and a transforming step of transforming the image datausing the second look up table; wherein: the predetermined logarithmcurve passes through a first point and a second point on a graph of aninput intensity versus an output intensity; the first point being atzero for the output intensity and a low value for the input intensity,the low value being a lowest value in a wider dynamic range thatcorresponds in the first look up table to a value higher than a lowernoise threshold; and the second point being at a maximum value in thefirst look up table for the output intensity and a high value for theinput intensity, the high value being a highest value in a wider dynamicrange that corresponds in the first look up table to a value lower thana higher noise threshold.